Sonification for Process Monitoring

ion led to users interpreting the representations in varied ways [87]. Furthermore, Kilander and Lönnqvist [75] warned that the “monitoring of mechanical activities such as network or server performance easily runs the risk of being monotonous” and Pedersen and Sokola reported that they soon grew tired of the highly abstract representations used in Aroma. It is interesting that they put some of the blame down to an impoverished aesthetic, feeling that involving expertise from the appropriate artistic communities would improve this aspect of their work. Cohen [32] identified a general objection to using audio for process monitoring: people in shared office environments do not want more noise to distract them. Buxton [26] argued that audio is ubiquitous and would be less annoying if people had more control over it in their environments. Lessons from acoustic ecology would be helpful here. Cohen [32] defined an acoustic ecology as “a seamless and information-rich, yet unobtrusive, audio environment”. Kilander and Lönnqvist [75] tackled this problem in their fuseONE and fuseTWO environments with the notion of a weakly intrusive ambient soundscape (WISP). In this approach the sound cues for environmental and process data are subtle and minimally-intrusive.16 Minimalor weak-intrusion is achieved in Kilander and Lönnqvist’s scheme by drawing upon the listener’s expectation, anticipation, and perception; anticipated sounds, say Kilander and Lönnqvist, slip from our attention. For example, a ticking clock would be readily perceived and attended to when its sound is introduced into the environment (assuming it is not masked by another sound). However, as the steady-state of the ticking continues and the listener expects or anticipates its presence its perceived importance drops and the sound fades from our attention [75]). However, a change in the speed, timbre, or intensity of the clock tick would quickly bring it back to the attention of the listener. Intrusiveness can thus be kept to a necessary minimum by using and modulating sounds that fit well with the acoustic ecology of the process monitor’s environment. The sonification is then able to be discriminated from other environmental sounds (either by deliberate attentiveness on the part of the listener, or by system changes to the sounds), yet is sufficiently subtle so as not to distract from other tasks that the listener (and others in the environment) may be carrying out. To increase the quality of the acoustic ecology further, Kilander and Lönnqvist used real-world sounds rather than synthesized noises and musical tones. They concluded that . . . easily recognisable and natural sounds . . . [stand] . . . the greatest chance of being accepted as a part of the environment. In particular, a continuous background murmur is probably more easily ignored than a singular sound, and it also continuously reassures the listener that it is operative. [75] Kilander and Lönnqvist’s weakly intrusive ambient soundscapes would thus seem to be a suitable framework for the design of peripheral process monitoring sonifications in which monitoring is not the user’s primary or sole task. Schmandt and Vallejo [101] noted the perception-distraction dichotomy. Their ListenIN system for monitoring activity in a domestic environment attempted to provide continuous but minimally-distracting awareness. Unfortunately, no formal studies have been carried out with the system to test this aim. Mynatt et al. [85] aimed with their Audio Aura 16Kilander and Lönnqvist actually used the adjective ‘non-intrusive’ to describe their sonifications. One could argue that this term is misleading as any sonification needs to be intrusive to some extent in order to be heard. Their term ‘weakly intrusive’ is more helpful and more accurate. Process Monitoring 475 scheme to provide environmental sonifications that enriched the physical world without being distracting. Tran and Myatt reduced the intrusiveness of their Music Monitor system [113] by overlaying earcons on top of music tracks chosen by the main user. The mixing of earcons with intentional music meant that other people in the environment would not be distracted as the encoded messages in the earcons would only be recognised by the main user: other people would just be aware of changes in the music. Of course, the fact that there is a music stream at all means that the system does still intrude into the environment. Rather, the attempt here was to minimize the negative distracting effects of that intrusion. Preliminary experimental results showed that the music was not distracting to those who were not monitoring the earcon messages embedded within it [113]. In a related project that combined auditory displays with tangible computing Bovermann et al. [16, 15] found that their Reim toolset could be used to provide peripheral monitoring without causing distraction or annoyance. Fatigue is sometimes mentioned as a potential problem associated with auditory display but it is notable that hearing is “more resistant to fatigue than vision” [91, p. 42] and so it is not clear that auditory displays should cause more problems in this regard than visual representations. 18.6.2 Emotive associations The degree and detail to which process data are sonified depends a great deal on the intended audience. Some may take a dispassionate view whilst others may attach emotional significance to the data. Cohen [31] found that it is difficult to construct “sounds which tell the right story and are also pleasant and emotionally neutral”. For example, in their work on sonifying weather reports Hermann, Drees, and Ritter [59] noted that whilst meteorologists would be interested in exploring and analyzing long-term time-series weather data, the average public consumer of a weather forecast requires a much more abstract view in terms of what will the weather be this afternoon, tomorrow, or at the weekend. Choice of activity and clothing are dependent on the weather, so a simple forecast indicating likely temperature, wind, and precipitation levels is sufficient for most tasks. Furthermore, a weather forecast can trigger an emotional response in the listener. Hermann, Drees, and Ritter put it that rather than having the detailed quantitative interest of the expert meteorologist, the “listener is concerned with [the weather’s] emotional value and contextual implications, which are not simply assessed from single . . . attributes like temperature or humidity in isolation” [59]. For instance, to a northern European a temperature of 30C would be very pleasant if the humidity is low, but quite unpleasant in overcast humid conditions. Thus, the sonification designer can take into account the fact that whilst the raw process data themselves are free of emotive content, their values and combinations can cause an inferential process of emotional coding on the part of the listener. Hermann, Drees, and Ritter [59] attempted to deal with this issue directly by deriving emotional relations from the high-dimensional ‘weather vector’. This was accomplished by constructing an Emo-Map (see Figure 18.6). The Emo-Map is a two-dimensional plot of hourly weather vectors: each vector is displayed as a single point on the plot. From this plot were derived a number of prototype weather states (such as ‘hot dry summer day’, ‘snowy winter day’ and ‘golden October day’). Each prototype had an associated emotive aspect (e.g., enervated and indifferent for the hot dry summer day, negative, calm, and indifferent for the snowy winter day, and positive emotional state for the